Trick Playback of Video Data

ABSTRACT

A portable device controlling playback of video data on a first device external to the portable device and including a browser module operative to present a display of a plurality of images, each image corresponding to a section of the video data, and to highlight an image corresponding to the section of the video data currently played back by the first device, a communication module coupled to the browser module and operative, in response to a selection of an image among the plurality of images by a user, to transmit a first command to the first device to play back the video data corresponding to the image selected by the user, wherein the browser module is further operative to adapt highlighting of the plurality of images in response to a second command received from said first device upon a change of the section of video data currently played back on the first device.

FIELD OF THE INVENTION

The present invention relates generally to video system, and moreparticularly, to method and apparatus of trick playback of video data.

BACKGROUND OF THE INVENTION

Digital video recording (DVR) technology has made it possible to providemore intelligent trick playback of video data, in addition to thetraditional time based trick mode. For example, advertisement zapping isproposed, allowing a user to skip a specific period if the video ispre-recorded and indexed with adequate meta-data.

There are various ways to insert resp. associate meta-data into resp.with a video stream. The automatic scene detection is a relevanttechnology, which can mark key scene changes of a video stream.Meta-data can be created for key scenes and inserted into a videostream.

The scene meta-data can be applied to support a scene-based trick mode.The scene-based trick mode concerns all playback control actions, suchas fast forward, rewind or slow playback, the control being based onscenes instead of timestamps. The advantage of scene-based trick mode isto allow moving quickly through a video without losing important scenes.

A DVD chapter book menu can perform a scene-based playback control.However, it only supports a “GOTO”-style function which allows theplayback to go to certain chapters/sessions of a movie. But this is nota replacement for the traditional trick mode.

A disadvantage of the above mentioned conventional scene-based trickmode is that it may confuse people who are used to the time-based trickmode through using traditional tools, such as a remote controller.

Another disadvantage arises from the lack of a mechanism for improvingthe trick mode creation.

SUMMARY OF THE INVENTION

According to one aspect of the invention, a method of controllingplayback of video data on a first device by a second device is provided.The method comprises, at the level of the second device, the steps of:presenting an image corresponding to a section of the video data; and inresponse to a change of the presentation status of the image,transmitting a command to the first device to adapt the playback statusof the section of video data on the first device as a function of thechange of the presentation status of the image.

According to another aspect of the invention, an apparatus forcontrolling playback of video data on another device is provided. Theapparatus comprises: means for displaying a set of images, each of whichcorresponds to a section of video data, in sequence of playback of videodata sections; means for highlighting the image during the time intervalof the playback of the corresponding section on the another device, andin response to a change of the displaying status of the image, fortransmitting a command to the another device to adapt the playbackstatus of the section of video data as a function of the change of thedisplaying status of the image.

BRIEF DESCRIPTION OF DRAWINGS

These and other aspects, features and advantages of the presentinvention will become apparent from the following description inconnection with the accompanying drawings in which:

FIG. 1 is a block diagram of the video system according to an embodimentof the present invention;

FIG. 2 is a high level flow chart showing the operation of the videosystem in FIG. 1;

FIG. 3 is a flow chart showing the operation of Video Decoder/TV moduleof the first device;

FIG. 4 is a flow chart showing the operation of the synchronizationmodule of the first device;

FIG. 5 is a flow chart showing the operation of communication module ofthe first device;

FIG. 6 is a flow chart showing the operation of the storybook browser ofthe second device;

FIG. 7 is a flow chart showing the operation of the synchronizationmodule of the second device;

FIG. 8 is a flow chart showing the operation of communication module ofthe second device;

FIGS. 9a and 9b show examples of exemplified user interface of storybookof a portable device;

FIG. 10 shows the detailed menu with control buttons of the portabledevice.

DETAIL DESCRIPTION OF PREFERRED EMBODIMENTS

In the following description, various aspects of an embodiment of thepresent invention will be described. For purposes of explanation,specific configurations and details are set forth in order to provide athorough understanding. However, it will also be apparent to one skilledin the art that the present invention may be practiced without thespecific details present herein.

In view of the foregoing disadvantages of the prior art, an embodimentof the present invention provides a video system that uses theinteraction with a visual media on a second device to realize the trickmode for the video playback on a first device.

Specifically, a visual media, which corresponds to a video clip or avideo section of the video playback on the first device, is presented onthe second device. The presentation status of the visual media on thesecond device is synchronized with the playback status of the video onthe first device. On one hand, the playback of a video section on thefirst device will lead to a presentation of the corresponding visualmedia on the second device. On the other hand, a status change of thevisual media on the second device will also result in a correspondingchange of the playback status of the video section on the first device.In this sense, there is a bi-directional synchronization between thepresentation status of the visual media on the second device and theplayback status of the video on the first device.

According to an embodiment of the invention, a video storybook is usedas the visual media on the second device for the purpose of implementinga scene-based trick mode. A storybook is a book-like media displaycontaining a collection of pages (slides, images), each of whichrepresents the corresponding video section in the video programdisplayed on the first device. In a sense, the video storybook is avideo summary of a video displayed on the first device for user tobrowse. The storybook is particularly beneficial to a mobile applicationsince there is no need to transmit a full version of video for the trickplay, which will save much bandwidth.

Next, an embodiment of a video system for implementing a scene-basedtrick playback on a first device by a storybook displayed on a seconddevice according to the principle of the invention will be described indetails.

FIG. 1 is a block diagram of the video system according to an embodimentof the present invention.

Referring to FIG. 1, the video system 10 comprises a head-end 11, afirst device 12 for video data playback, and a second device 13 forpresenting a storybook of the video data of the first device. Thehead-end 11, the first device 12 and the second device 13 are in signalcommunication via a communication network (CN) 14.

The first device 12 may include but is not limited to a Set Top Box(STB) or a TV set. The second Device 13 may include but is not limitedto a portable device, a tablet, a PC or a laptop. The CN 14 may be anytype that has the capability of carrying and transporting video content,such as broadcast network, IP network, or a combination thereof.

The head-end 11 comprises: a Video Content Server (VCS) 110 forproviding video content; a Video Broadcasting Server (VBS) 111 forreceiving the video content from VCS 110 and broadcasting to the CN 14such as a cable network; a Storybook Creator (SC) 112 for creatingstorybooks for the video content from the VCS 110 based on a pre-definedrule (PR) 113. As shown in FIG. 1, the head-end 11 can also comprise aUser Action Collector (UAC) 114 for collecting from the CN 14 andanalyzing the action statistics that users performed on the storybook,and for sending the analysis results to SC 112 which can createstorybooks based on the analysis results from UAC 114 and/or the abovedescribed pre-defined rule. As shown in FIG. 1, it can be appreciatedthat the SC 112 may also offer an editor interaction UI, through whichan editor can create a storybook manually. The created storybooks canalso be stored in a Storybook Inventory (SI) 115 before they are sent tothe CN 14.

It should be noted in a case that the CN 14 in the system does notprovide a return channel (e.g., in the case of a broadcast network), nouser action statistics will be collected by the UAC 114.

The first device 12 comprises: a Video Decoder/TV (VD/TV) 121 fordecoding the video from the CN 14 and performing video playback thereon;a Synchronization Module (SM) 122 for synchronizing the status of thevideo playback on the first device 12 to the display status of storybookon the second device 13; and a Communication Module (CM) 123 forcommunicating with the second device 13.

It can be appreciated that the status of video playback on the VD/TV 121can be controlled by a remote controller (not shown) or commands from SM122 of the first device 12.

The SM 122 will report the status of video playback on the VD/TV 121 ofthe first device 12 via the CM 123 to the second device 13 as areference of synchronization of the status of the storybook displaythereon. Specifically, the report can be time driven or event driven.That is, on one hand, the SM 22 will send reports periodically to thesecond device 13 (we can call it a heartbeat mechanism), informing ofthe progress of the video playback on the VD/TV 121 of the first device12 for the second device 13 to synchronize on. The period of the reporttransmission can be predefined based on the requirements of theapplication. On the other hand, the SM 122 will detect if there are anychanges in the status of the video playback on the VD/TV 121, forexample, according to commands from the remote controller (as anexample, if the user performs a fast forward action on remote controllerwhich results in the progress change of video playback on VD/TV 121, theVD/TV 121 will immediately notify SM 122 about it by sending a messageto SM122). If yes, the SM 122 will immediately transmit a report of thecurrent video playback status of the first device 12 to the seconddevice 13 through the CM 123 for the second device 13 to correspondinglychange (synchronize) the status of the storybook display thereon.

As to be described later, the SM 122 will also receive a report via theCM 123 from the second device 13 on the current status of storybookdisplay and send a command to the VD/TV 121, requesting the progress orstatus of the video playback on the VD/TV 121 to be synchronized withthat of the storybook display on the second device 13.

The second device 13 comprises: a Storybook Browser (SB) 131 fordisplaying a storybook; a Synchronization Module (SM) 132 forsynchronizing the status of the storybook display on the second device13 with the status of the video playback on the first device 12; and aCommunication Module (CM) 133 for communicating with the first device12.

The storybook can be created and transmitted to the second device 13online or offline depending on the requirements of the video service.The head-end 11 can directly transmit the storybook to the second device13 independently from the video content data. Alternatively, thestorybook can be embedded within the video stream under the form ofmetadata or some marks at the head-end 11 (for example, in term of anMPEG-2 video system, the adaptation field in the MPEG-2 transport streamcan be used to carry the link of storybook thanks to the availablereserved bytes in the adaptation field) and transmitted to the firstdevice 12 which will then extract the storybook relevant data andtransmit it to the second device 13 to compose the storybook proper,using the metadata and video stream data (if appropriate e.g. to extractrepresentative images). The storybook can be transmitted either in aself-contained fashion (the entire storybook data is transmitted) or byreference through links to data to be collected. In the latter case, thesecond device 13 can fetch the storybook data by accessing anappropriate server using the links.

As indicated earlier, the storybook may also be stored in the SI 115 ofthe head-end 11. In this case, a user can access the storybook byaccessing the SI 115, e.g. through using an appropriate IP address. ThisIP address may be fixed and not necessarily sent with the video stream.When the user browses the storybooks and selects one of them, thecorresponding video will be streamed from the head-end 11 to the firstdevice 12.

The storybook can also be offered to users coupled with video on demand(VoD) or time-shift TV services.

Users may store video on the first device 12 coupled with thecorresponding storybook by taking advantage of a personal video recorder(PVR) function.

According to a variant embodiment, if the CN 14 in FIG. 1 has a returnchannel, the user interaction with the storybook can be sent to the UAC114 of the head-end 11 to facilitate and enrich the storybook creation.For example, the everyday click-through rates of the news video sectionscan be used for news selection in order to compose the weekly newsstorybook.

As mentioned above, upon receipt of a report from the first device 12 onthe progress or status of the video playback, the SM 132 willsynchronize the progress or status of the storybook display on the SB131 to that of the video playback on the first device 12.

The SM 132 will report the status of the storybook display on the SB 131of the second device 13 via the CM 133 to the first device 12 as areference of synchronization of the status of the video playbackthereon. In particular, the SM 132 will detect the status of thestorybook display on the SB 131 of the second device 13. If any changesin the status are detected, the SM 132 will immediately report thecurrent status of the storybook display of the second device 13 to thefirst device 12 through the CM 133.

The status of the storybook display on the SB 131 of the second device13 can be changed or controlled according to the user interaction to thesecond device 13.

SM 132 can also periodically send a report on the status of thestorybook display on the second device 13 to the first device 12 inorder to facilitate synchronization between the two devices.

Next, an embodiment of the operation procedure of the video system inFIG. 1 will be described in detail with reference to the FIGS. 2-8.

FIG. 2 is a high level flow chart showing the operation of the videosystem in FIG. 1.

As shown in FIG. 2, at the step S201, the head-end 11 will create astorybook, for example, based on a predefined rule and user interactionstatistics.

At the step S202, the head-end 11 sends video content together withcorresponding the storybook (or its link) to the CN 14.

At the step S203, the first device 12 will receive video content and thecorresponding storybook (or its link) from the CN 14.

At the step S204, the first device 12 will decode and display videocontent on its screen and distribute the storybook (or its link) to thesecond device 13.

At the step S205, in case the second device 13 receives a link for thestorybook, it will download the storybook from the head-end 11.

At the step S206, the first device 12 will receive commands for examplethrough a remote controller, and the second device 13 will allow theuser to interact with the storybook thereon.

At the step S207, status synchronization will be performed between thevideo playback on the first device 12 and the storybook display on thesecond device 13. In this step, a report on user interactions can alsobe sent to the head-end 11 for improving the storybook creation in thefuture.

FIG. 3 is a flow chart showing the operation of Video Decoder/TV module121 of the first device 12.

As shown in FIG. 3, at the step S301, the Video Decoder/TV module 121receives video data from the CN 14.

At the step S302, the Video Decoder/TV module 121 decodes and plays backthe video data on the first device 12.

At the step S303, the Video Decoder/TV module 121 will determine whethera command is received, for example, from a remote controller.

If the result of the step S303 is “No”, then the procedure will proceedto the step S304 where the Video Decoder/TV module 121 will furtherdetermine whether a command is received from the SM 122. If the resultof the step S304 is “Yes”, the procedure will proceed to the step S305where the Video Decoder/TV module 121 will correspondingly change videoplayback status based on the command from the SM122. Otherwise, theprocedure will return to the step S302.

If the result of the step S303 is “Yes”, then the procedure will proceedto the step S306 where the Video Decoder/TV module 121 will detectwhether a STOP command is received.

If the result of the step S306 is “No”, then the procedure will proceedto the step S307 where the Video Decoder/TV module 121 will change thevideo playback status and send a status report to the SM 122. If theresult of the step S306 is “Yes”, then the procedure will proceed to thestep S308 where the Video Decoder/TV module 121 will stop video playbackand send a status report to the SM 122.

FIG. 4 is a flow chart showing the operation of the SM 122 of the firstdevice 12.

As shown in FIG. 4, at the step S401, the SM 122 starts to synchronizethe video playback on the first device 12 with the storybook display onthe second device 13.

At the step S402, the SM 122 determines whether it is time to send out aperiodical report.

If the result of the step S402 is “Yes”, at the following step S403, theSM 122 will retrieve the video playback status from the Video Decoder/TVmodule 121, compose the status report and send the report to the seconddevice 13 through the CM 123.

If the result of the step S402 is “No”, the procedure will proceed tothe step S404 where the SM 122 will determine whether a report isreceived from the Video Decoder/TV module 121.

If the result of the step S404 is “Yes”, the procedure will proceed tothe step S409 where the SM 122 will send the report to the second device13 through the CM 123.

If the result of the step S404 is “No”, at the following step S405, theSM 122 will determine whether a report is received from the seconddevice 13 through the CM 123.

If the result of the step S405 is “No”, the procedure will return to thestep S402. If the result of the step S405 is “Yes”, at the followingstep S406, the SM 122 will retrieve the current status of video playbackon the first device 11 from the Video Decoder/TV module 121.

At the following step S407, the SM 122 will determine whether it needsto change video playback status. If the result of the step S407 is “No”,the procedure will return to the step S402. If the result of the stepS407 is “Yes”, at the step S408, the SM 122 will compose and send acommand to the Video Decoder/TV module 121 for changing video playbackstatus.

FIG. 5 is a flow chart showing the operation of CM 123 of the firstdevice 12.

As shown in FIG. 5, at the step S501, the CM 123 starts communication.

At the step S502, the CM 123 will wait for a status report from theSM122 or the second device 13.

At the step S503, the CM 123 will determine whether a report is receivedfrom the SM 122. If the result of the step S503 is “Yes”, the procedurewill proceed to the step S506 where the CM 123 will send the report tothe second device 13. If the result of the step S503 is “No”, theprocedure will proceed to the step S504 where the CM 123 will determinewhether a report is received from the second device 13.

If the result of the step S504 is “Yes”, the procedure will proceed tothe step S505 where the CM 123 will send the report to the SM122. If theresult of the step S504 is “No”, the procedure will return to the stepS502.

FIG. 6 is a flow chart showing the operation of the SB 131 of the seconddevice 13.

As shown in FIG. 6, at the step S601, the SB 131 receives the storybookfrom the CN 14.

At the step S602, the SB 131 displays the storybook on the second device13.

At the step S603, the SB 131 will determine whether a command isreceived from the user.

If the result of the step S603 is “No”, then the procedure will proceedto the step S604 where the SB 131 will further determine whether acommand is received from the SM 132. If the result of the step S604 is“Yes”, the procedure will proceed to the step S605 where the SB 131 willcorrespondingly change the storybook playback status based on thecommand from the SM132. Otherwise, the procedure will return to the stepS602.

If the result of the step S603 is “Yes”, then the procedure will proceedto the step S606 where the SB 131 will detect whether a STOP command isreceived.

If the result of the step S606 is “No”, then the procedure will proceedto the step S607 where the SB 131 will change storybook playback statusand send status report to the SM 132. If the result of the step S606 is“Yes”, then the procedure will proceed to the step S608 where SB 131will close the storybook and send a status report to the SM 132.

FIG. 7 is a flow chart showing the operation of the SM 132 of the seconddevice 13.

At the step S701, the SM 132 starts to synchronize the storybook displayon the second device 13 with the video playback on the first device 12.

At the step S702, the SM 132 determines whether it is time to send out aperiodical report.

If the result of the step S702 is “Yes”, at the following step S703, theSM 132 will retrieve the video playback status from the SB 131, composethe status report and send the report to the first device 12 through theCM 133.

If the result of the step S702 is “No”, the procedure will proceed tothe step S704 where the SM 132 will determine whether a report isreceived from the SB 131.

If the result of the step S704 is “Yes”, the procedure will proceed tothe step S709 where the SM 132 will send the report to the first device12 through the CM 133.

If the result of the step S704 is “No”, at the following step S705, theSM 132 will determine whether a report is received from the first device12 through the CM 133.

If the result of the step S705 is “No”, the procedure will return to thestep S702. If the result of the step S705 is “Yes”, at the step S706,the SM 132 will retrieve the current status of storybook playback on thesecond device 13 from the SB 131.

At the step S707, the SM 132 will determine whether it needs to changestorybook playback status. If the result of the step S707 is “No”, theprocedure will return to the step S702. If the result of the step S707is “Yes”, at the step S708, the SM 132 will compose and send a commandto the SB 131 for changing storybook playback status.

FIG. 8 is a flow chart showing the operation of CM 133 of the seconddevice 13.

As shown in FIG. 8, at the step S801, the CM 133 starts communication.

At the step S802, the CM 133 will wait for a status report from the SM132 or the first device 12.

At the step S803, the CM 133 will determine whether a report is receivedfrom the SM 132. If the result of the step S803 is “Yes”, the procedurewill proceed to the step S806 where the CM 133 will send the report tothe first device 12. If the result of the step S803 is “No”, theprocedure will proceed to the step S804 where the CM 133 will determinewhether a report is received from the first device 12.

If the result of the step S804 is “Yes”, the procedure will proceed tothe step S805 where the CM 133 will send the report to the SM132. If theresult of the step S804 is “No”, the procedure will return to the stepS802.

An exemplified user interface of a storybook is shown in FIG. 9. In thisexample, a trick display of video data on a STB/TV is implemented by theinteraction between the TV and a portable device.

As shown in FIG. 9a , the video data, for example a movie, is displayedon the TV screen 901. It can be appreciated that the movie data iscomposed of a plurality of sections, each of which can be represented bya key scene (which is a fixed image according to the present embodiment,but which may be a video sequence or another representative item). Whena video section i (where i may be 0, 1, 2, . . . ) is played back on theTV screen 901, a corresponding key scene i will be displayed on thescreen of the portable device 902. In this case, we may call the keyscene i the page i since it looks like one page of a storybook which isdisplayed on the portable device 902 to describe the content played backon the TV screen 901.

As an alternative, a matrix of several key scenes (pages of thestorybook) in sequence can be presented on the screen of the portabledevice 902, as shown in FIG. 9b . In this case, when a video section iis played back on the TV screen 901, the corresponding key scene i willbe highlighted dynamically (indicated in FIG. 9b by the block with thicklines).

It can be appreciated by a person skilled in the art that the above twotypes of user interface can be combined into one implementation. Thatis, when a video section i is played back on the TV screen 901, acorresponding key scene i will be displayed on the screen of theportable device 902. And at the same time, a list of several key scenesin sequence can also be presented on an appropriate position of thescreen of the portable device 902, with the corresponding key scene Ihighlighted.

An example specifying the content of a storybook is given by thefollowing code.

<?xml version=“1.0” encoding=“UTF-8”?>-<storybookxmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”xsi:noNamespaceSchemaLocation=“storybook.xsd”>

<page id=“0” timestamp=“30.08” image_src=“assets/0.png”image_thumbnail_src=“assets/0_thumbnail.png”/>

<page id=“1” timestamp=“114.032” image_src=“assets/1.png”image_thumbnail_src=“assets/1_thumbnail.png”/>

<page id=“2” timestamp=“299.04” image_src=“assets/2.png”image_thumbnail_src=“assets/2_thumbnail.png”/>

<page id=“3” timestamp=“399.84” image_src=“assets/3.png”image_thumbnail_src=“assets/3_thumbnail.png”/>

<page id=“4” timestamp=“580.16” image_src=“assets/4.png”image_thumbnail_src=“assets/4_thumbnail.png”/>

<page id=“5” timestamp=“725.2” image_src=“assets/5.png”image_thumbnail_src=“assets/5_thumbnail.png”/>

<page id=“6” timestamp=“919.12” image_src=“assets/6.png”image_thumbnail_src=“assets/6_thumbnail.png”/>

<page id=“7” timestamp=“1024.04” image_src=“assets/7.png”image_thumbnail_src=“assets/7_thumbnail.png”/>

<page id=“8” timestamp=“1207.6” image_src=“assets/8.png”image_thumbnail_src=“assets/8_thumbnail.png”/>

<page id=“9” timestamp=“1401.16” image_src=“assets/9.png”image_thumbnail_src=“assets/9_thumbnail.png”/>

<page id=“10” timestamp=“1548.56” image_src=“assets/10.png”image_thumbnail_src=“assets/10_thumbnail.png”/>

<page id=“11” timestamp=“1689.56” image_src=“assets/11.png”image_thumbnail_src=“assets/11_thumbnail.png”/>

</storybook>

In this example, the page id denotes the page index; the timestampspecifies the time relationship between the page and the correspondingvideo content; the image_src indicates the picture link of the page; andthe image_thumbnail indicates the thumbnail picture link of the page.

As shown above, the storybook contains 12 pages with the range from 0 to11. Page 0 corresponds to the movie section 0 with the timestamp rangingfrom 0 to 30.08 seconds, page 1 corresponds to the movie section 1 withthe timestamp ranging from 30.09 to 114.032 seconds, and so on.

According to the present embodiment, the status of the movie playback onthe TV screen is “bi-directionally” synchronized with the display statusof the scenes on the screen of a portable device 902.

Specifically, on one hand, when the movie is played back on the TV, acorresponding page (i.e., key scene) on the portable device 902 will bedynamically highlighted during a time interval defined by the timestamprange of a video section to reflect the movie playback progress on theTV. With forward/backward playback of the movie, the highlighted page ofthe storybook shall be updated as well in order to ensure the abovedescribed synchronization. If the user uses a traditional remotecontroller to fast forward/backward the movie on the TV, thecorresponding progress will also be reflected on the portable device 902with the highlight moving forward/backward page by page.

On the other hand, user interactions to the pages (key scenes) displayedon the screen of the portable device 902 can realize a scene-based trickplayback of the movie on the TV. For example, the user can click any ofthe pages to immediately switch to the corresponding movie section onthe TV. In addition, the portable device 902 can also be provided with amenu having several functional buttons to facilitate various trickplayback operations.

The detailed menu with control buttons of the portable device is shownin FIG. 10. The following are some exemplary trick playback functionsprovided by the portable device:

GOTO: by tapping/selecting a page in storybook the user can cause videoplayback to start from the corresponding scene.

NEXT/PREV: by swiping left, respectively right to next respectivelyprevious page in storybook can cause video playback to start from thebeginning of the next respectively previous scene.

FF/RW: by swiping repeatedly left respectively right on the storybook,the user can cause video to fast forward/rewind at a speed proportionalto the repeating frequency. The speed might be measured in terms ofnumber of scenes per second.

PLAY/PAUSE: by tapping on PLAY, the user can cause video playback tostart; maintaining the finger on a page in storybook can cause videoplayback to pause on a still image of the corresponding scene.

REPEAT: by circling on a page in storybook, the user can cause repeatedplayback of the corresponding section.

SLOW: by slowly sliding on a page in storybook, the user can cause slowplayback of the corresponding section.

SCOPE: by selecting a start page and an end page in a storybook, theuser can make video start and stop at the corresponding scenes.

ZOOM: by zooming/focusing on an image page of a storybook, focused zoomis carried out on the video being played back, the center of the zoomcan be moved around the display.

Message exchange between the STB/TV and the portable device can betriggered by any one of the following events:

any operation on the STB/TV through the remote controller;

any user interaction performed on the storybook;

a periodical report on the STB/TV video status; and

a periodical report on the storybook status of the portable device.

The message exchange between the STB/TV and the portable device is toreport local status between devices, and therefore to achievesynchronization as described above. The following message format can beused:

Availability Message Payload Comments Required { Start of the messagepayload Required “Protocol name”: Indicate the protocol name“STORYBOOK”, Required “Version”: “1.0”, Indicate the protocol versionRequired “Message ID”: “1”, Indicate the message identity Required“Message name”: “STATUS Indicate the message name REPORT”, Required“Device name”: “STB/TV”, Indicate the device name, such as STB/TV orPortable device/Storybook. Required “Device Status”: “Active”, Indicatethe current device status such as PowerOn, Active and PowerOff Required“Media status”: “Play”, Current media status such as: Goto, FastForward,Rewind, Prev, Next, Play/Pause, Stop, Repeat, Slow, Scope, etc. Optional“Media type”: “Video”, Indicate the current media's type Optional “Mediaformat”: “MPEG4”, Indicate the current media's coding standard Optional“Media source type”: “TV”, The media source type indicates whether ornot the media is TV, PVR, or Streaming Optional “Media url”: “22”,Indicate the link information for media flow, which is the channel IDfor the TV, directory and file name for the PVR, and url for thestreaming media Required “Current progress”: 235690 The current timestamp for the media: the updated video time stamp if the device name isSTB/TV; the time stamp of starting scene corresponding to the currentpage if the device name is Portable device/Storybook Required } End ofthe message payload

Note the message body can be extended to include more fields accordingto the needs of application.

An example of a message exchange between the portable device and theSTB/TV when a user presses PREV button on the portable device will bedescribed. In this case, the highlighted page of storybook goes to theprevious page.

When the user presses the PREV button, a report is sent from theportable device to the STB/TV with following message body to inform of achange of display status of the storybook on the portable device.

Availability Message Payload Comments Required { Start of the messagepayload Required “STORYBOOK”, Indicate the protocol name Required “1.0”,Indicate the protocol version Required “1”, Indicate the messageidentity Required “STATUS REPORT”, Indicate the message name Required“Portable device/ Indicate the device name, such Storybook”, as STB/TVor Portable device/Storybook. Required “Active”, Indicate the currentdevice status such as PowerOn, Active and PowerOff Required “Prev”,Current media status such as: Goto, FastForward, Rewind, Prev, Next,Play/Pause, Stop, Repeat, Slow, Scope, etc. Optional “Video”, Indicatethe current media's type Optional “MPEG4”, Indicate the current media'scoding standard Optional “TV”, The media source type indicates whetheror not the media is TV, PVR, or Streaming Optional “Media url”: “22”,Indicate the link information for media flow, which is the channel IDfor the TV, directory and file name for the PVR, and url for thestreaming media Required “Current progress”: 30.09 The current timestamp for the media: the updated video time stamp if the device name isSTB/TV; the time stamp of starting scene corresponding to the currentpage if the device name is Portable device/Storybook Required } End ofthe message payload

After receiving the status report from the portable device, the STB/TVwill perform the above described synchronization with the information inthe message. In addition, the STB/TV can also send a confirmationmessage back to the portable device with the following message body.

Availability Message Payload Comments Required { Start of the messagepayload Required “STORYBOOK”, Indicate the protocol name Required “1.0”,Indicate the protocol version Required “1”, Indicate the messageidentity Required “STATUS REPORT Indicate the message nameCONFIRMATION”, Required “STB/TV”, Indicate the device name, such asSTB/TV or Portable device/Storybook. Required “Active”, Indicate thecurrent device status such as PowerOn, Active and PowerOff Required“Prev”, Current media status such as: Goto, FastForward, Rewind, Prev,Next, Play/Pause, Stop, Repeat, Slow, Scope, etc. Optional “Video”,Indicate the current media's type Optional “MPEG4”, Indicate the currentmedia's coding standard Optional “TV”, The media source type indicateswhether or not the media is TV, PVR, or Streaming Optional “Media url”:“22”, Indicate the link information for media flow, which is the channelID for the TV, directory and file name for the PVR, and url for thestreaming media Required “Current progress”: 30.09 The current timestamp for the media: the updated video time stamp if the device name isSTB/TV; the time stamp of starting scene corresponding to the currentpage if the device name is Portable device/Storybook Required } End ofthe message payload

It is to be understood that numerous modifications may be made to theillustrative embodiments and that other arrangements may be devisedwithout departing from the spirit and scope of the present invention asdefined by the appended claims.

1. A portable device for controlling playback of video data on a firstdevice external to said portable device, comprising: a browser moduleoperative to present a display of a plurality of images, each imagecorresponding to a section of the video data, and to highlight an imagecorresponding to the section of the video data currently played back bythe first device; a communication module coupled to the browser moduleand operative, in response to a selection of an image among theplurality of images by a user, to transmit a first command to the firstdevice to play back the video data corresponding to the image selectedby the user; and wherein the browser module is further operative toadapt highlighting of the plurality of images in response to a secondcommand received from said first device upon a change of the section ofvideo data currently played back on the first device.
 2. The portabledevice of claim 1, wherein the plurality of images differs from thevideo data.
 3. The portable device of claim 1, wherein the plurality ofimages comprises a summary of a story in the video data.
 4. The portabledevice of claim 1, wherein each section of the video data comprises ascene of the video data.
 5. The portable device of claim 1, wherein thecommunication module being further operative to receive the plurality ofimages from the second device.
 6. The portable device of claim 1,wherein the communication module being further operative to send to ahead-end a report on an interaction of the user with the plurality ofimages, wherein the report includes a click-through rate of video datasections used to compose the plurality of images to provide a summary ofthe video data sections.